1. Technical Field
The present inventive subject matter relates to image recognition technologies associated with circular plots, including genomic circular plots.
2. Background
The background description includes information that may be useful in understanding the present inventive subject matter. It is not an admission that any of the information provided herein is prior art or applicant admitted prior art, or relevant to the presently claimed inventive subject matter, or that any publication specifically or implicitly referenced is prior art or applicant admitted prior art.
Circular plots have grown in popularity within the scientific communities. Circular plots are able to condense an extremely large amount of information into an ink efficient and space efficient visual package. There are numerous circular plotting packages available, including the Circos® package that pioneered the effort, which can be found at URL circos.ca (Martin I Krzywinski, Jacqueline E Schein, Inanc Birol, Joseph Connors, Randy Gascoyne, Doug Horsman, Steven J Jones, and Marco A Marra, Genome Res. Published in Advance Jun. 18, 2009, doi:10.1101/gr.092759.109).
Circular plots can vary widely depending on the information they are required to present. Examples of fairly simple circular plots can be found in U.S. Patent Application Pub. No. 2015/0031641 to Ross L. Levine et al., entitled “Methods and Compositions for the Diagnosis, Prognosis and Treatment of Acute Myeloid Leukemia” (“the '641 publication”). See, for example, FIGS. 1a-1c, 6a-6n, 7a-7c, and 8a of the '641 publication. As described in paragraph [0032] of the '641 publication, FIG. 1 is a Circos® diagram depicting relative frequency and pairwise co-occurrence of mutations in de novo AML patients enrolled in the ECOG protocol E1900 (Panel A), in which the arc length corresponds to the frequency mutations in the first gene and the ribbon width corresponds to the percentage of patients that also have a mutation in the second gene. Further, circular plots can also become quite complex representing dense, rich information. An example of a relatively complex circular plot can be found in U.S. Patent Application Pub. No. 2014/0115515 to Julie Adams et al., entitled “Genome Explorer System to Process and Present Nucleotide Variations in Genome Sequence Data” (“the '515 publication”). See, for example, FIG. 3A of the '515 publication. As described in paragraph [0064] of the '515 publication, FIG. 3A is an overview display of the entire genome sequence of a patient sample in the form of a circular or Circos®-style plot, in which the frequency of structural variations at locations across the chromosome map are shown and curved lines in the middle show apparent interchromosomal junctions.
Circular plots similar to the one depicted in FIG. 3A of the '515 publication are typically used to present detailed information to one or more stakeholders. Typically, such plots are incorporated into scientific presentations. However, the circular plots can also be used by one or more healthcare providers. For example, a doctor might request that a cancer patient obtain a GPS Cancer™ test, such as those offered by NantHealth, Inc. (see URL www.gpscancer.com). One possible result from the test could include a detailed circular plot showing a condensed representation of a whole genome sequence of the patient's tumor.
Interestingly, complex genomic circular plots have disadvantages in view of their ability to present a vast amount of information in compact form. On one hand, a stakeholder is able to assess quickly an overview of a patient's genomic status based on the observed information. However, on the other hand, the stakeholder would have to pour over the circular plot to find detailed information, which is time consuming. Further, the stakeholder lacks the ability to compare one circular plot to other similar circular plots in order to make detailed comparisons relating to treatment, diagnosis, or prognosis.
It is possible to leverage existing image recognition technologies that could “recognize” an image of a circular plot among many plots. For example, a circular plot could be analyzed by an implementation of a scale invariant feature transform (SIFT) algorithm as described in U.S. Pat. No. 6,711,293 to Lowe titled “Method and Apparatus for Identifying Scale Invariant Features in an Image and Use of Same for Locating an Object in an Image,” filed on Mar. 6, 2000, the content and substance of which is incorporated herein by reference. The algorithm yields one or more SIFT descriptors, which can then be used as an index to look up information about the recognized plot. Other examples of descriptors include those described in U.S. Pat. No. 8,866,924 to Tang et al. titled “Local Image Feature Descriptors According to Circular Distribution Information,” filed Oct. 28, 2011, the content and substance of which is incorporated herein by reference, and U.S. Pat. No. 9,412,176 to Song et al. titled “Image-based Feature Detection using Edge Vectors,” filed May 6, 2015, the content and substance of which is incorporated herein by reference.
Searching based on salient parameters is described in U.S. Pat. No. 7,016,532 to Boncyk et al. titled “Image Capture and Identification Systems and Process,” filed on Nov. 5, 2001, the content and substance of which is incorporated herein by reference; U.S. Pat. No. 7,477,780 to Boncyk et al. also titled “Image Capture and Identification Systems and Process,” filed internationally on Nov. 5, 2002, the content and substance of which is incorporated herein by reference; and U.S. Pat. No. 7,680,324 to Boncyk et al. titled “Use of Image-Derived Information as Search Criteria for Internet and Other Search Engines,” filed on Aug. 15, 2005, the content and substance of which is incorporated herein by reference. While these computer-based techniques provide utility with respect to returning indexed information about an a priori known image or an a priori known object in an image, they would fail or lack efficiency with respect to providing an actual interpretation of the information contained in a complex genomic plot that is newly generated or not yet known.
At the other end of the spectrum, techniques such as those employed for reading bar codes yield exact interpretations. However, especially in the case of complex genomics circular plots, it is not yet possible for a computing device to provide an exact interpretation of the data represented in the circular plot for multiple reasons. One example difficulty is that the circular plots can include a fine level of detail that cannot necessarily be captured by an imaging device. The loss of fidelity in the captured image results in loss of information during analysis. Another example difficulty is that there are no standard definitions for genomic circular plots by which such plots can be interpreted. Thus, it is not necessarily possible for a device to a priori know what type of plot it is viewing in order to generate a meaningful interpretation.
Even beyond the difficulties associated with a computing device interpreting the information in a genomic circular plot, stakeholders such as healthcare providers would benefit from initiating transactions from captured images of such circular plots. For example, a doctor would benefit from quickly capturing an image of a patient's genomic circular plot and then initiating a prescription or matching the patient to clinical trials, just to name a few benefits. Thus, there remains a considerable need for technologies to convert observed circular plots into meaningful actions.
All publications identified herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies and the definition of that term in the reference does not apply.